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1. The designated Office is hereby notified of its election made: 

[x] in the demand filed with the International Preliminary Examining Authority 

05 June 2000 (05.06.00) 



I I in a notice effecting later election filed with the International Bureau on: 



2. The election was 

I I was not 

made before the expiration of 1 9 months from the priority date or, where Rule 32 applies, within the time limit under 



Rule 32.2(b). 
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International publication date (day/month/year) 
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Priority date (day/month/year) 
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1. The applicant is hereby notified of the date of receipt (except where the letters "NR" appear in the right-hand colunnn) by the 
International Bureau of the priority document(s) relating to the earlier application(s) indicated below. Unless otherwise 
indicated by an asterisk appearing next to a date of receipt or by the letters "NR", in the right-hand column, the priority 
document concerned was submitted or transmitted to the International Bureau in compliance with Rule 17,l(a) or (b). 


2. This updates and replaces any previously issued notification concerning submission or transmittal of priority documents. 


3. An asterisM*) appearing next to a date of receipt, in the right-hand column, denotes a priority document submitted 
or transmitted to the International Bureau but not in compliance with Rule 17.1 (a) or (b). In such a case, the attention 
of the applicant is directed to Rule 17.1(c) which provides that no designated Office may disregard the priority claim 
concerned before giving the applicant an opportunity, upon entry into the national phase, to furnish the priority document 
within a time limit which is reasonable under the circumstances. 


4. The letters "NR" appearing in the right-hand column denote a priority document which was not received by the International 
Bureau or which the applicant did not request the receiving Office to prepare and transmit to the International Bureau, 
as provided by Rule 17.1 (a) or (b), respectively. In such a case, the attention of the applicant is directed to Rule 17.1(c) which 
provides that no designated Office may disregard the priority claim concerned before giving the applicant an opportunity, 
upon entry into the national phase, to furnish the priority document within a time limit which is reasonable under the 
circumstances. 
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4th Floor 

1755 Jefferson Davis Highway 
Arlington, VA 22202 
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International filing date (day/month/year) 
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Priority date (day/month/year) 

05 November 1998 (05.11.98) 
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ARCH DEVELOPMENT CORPORATION et a! 



1. Notice is hereby given that the International Bureau has communicated, as provided in Article 20, the international application 
to the following designated Offices on the date indicated above as the date of mailing of this Notice: 

AU,JP,US 



In accordance with Rule 47.1 (c), third sentence, those Offices will accept the present Notice as conclusive evidence that 
the communication of the international application has duly taken place on the date of mailing indicated above and no copy 
of the international application is required to be furnished by the applicant to the designated Office(s). 

2. The following designated Offices have waived the requirement for such a communication at this time: 
CA,EP 



The communication will be made to those Offices only upon their request. Furthermore, those Offices do not require the 
applicant to furnish a copy of the international application (Rule 49.1 (a-bis)). 

3. Enclosed with this Notice is a copy of the international application as published by the International Bureau on 
18 May 2000 (18.05.00) under No. WO 00/28466 

REMINDER REGARDING CHAPTER II (Article 31(2)(a) and Rule 54.2) 

If the applicant wishes to postpone entry into the national phase until 30 months (or later in some Offices) from the priority 
date, a demand for international preliminary examination must be filed with the competent International Preliminary 
Examining Authority before the expiration of 19 months from the priority date. 

It is the applicant's sole responsibility to monitor the 1 9-month time limit. 

Note that only an applicant who is a national or resident of a PCT Contracting State which is bound by Chapter II has the 
right to file a demand for international preliminary examination. 



REMINDER REGARDING ENTRY INTO THE NATIONAL PHASE (Article 22 or 39(1)) 

If the applicant wishes to proceed with the international application in the national phase, he must, within 20 months 
or 30 months, or later in some Offices, perform the acts referred to therein before each designated or elected Office. 

For further important information on the time limits and acts to be performed for entering the national phase, see the 
Annex to Form PCT/IB/301 (Notification of Receipt of Record Copy) and Volume II of the PCT Applicant's Guide. 
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0730006520W'O 


FOR FURTHER ACTION See Notification of Transmittal of International 

Preliminary Examination Report (Form PCT/IPEA/4I6) 


International application No. 
PCT/US99/24007 


International filing date 


Priority date (day/month/year) 
05 NOVEMBER 1998 


International Patent Classil ication (IPC) or national classiHcation and IPC 
Please See Supplemental Sheet. 


Applicant 

ARCH DEVELOPMENT CORPORATION 



This international preliminary examination report has been prepared by this International Preliminary 
Exammmg Authority and is transmitted to the applicant according to Article 36. 

This REPORT consists of a total of sheets. 

I I ' ' ^'I^^^ accompanied by ANNEXES, i.e., sheets oi'the description, claims and/or drawings which have 

been amended and are the basis for this report and/or sheets containing rectifications made before this Authority 
(.see Rule 70.16 and Section 607 of the Administrative In.structions under the PCT). 

These ^inne.Ncs ctJiisisl of a total of 

O 

sheets. 



3. This report contains indications relating to the following items: 
1 [ x| t^^-^is of the report 
II Q Priority 

III Q Non-cslablishment of report with regard to novelty, inventive step or indusUial applicability 

IV I I Lack of unity of invention 

V Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability- 
citations and explanations supporting such statement 

VI 1^ Certa in documents cited 

VII I I Cen;iin defects in the international application 
IZH ^^b.^ci-valions on the international applic 



ication 



Date of submission of the demand 
05 JUNE 2()00 


Date of completion of this report 
08 MAY 2001 \ 


Name and mailing address of the 1 PEA/ US 

Commissioner of Patents ;md Trademarks 
Box PCT 

Washington. D C. 2n2.-l 

Facsimile No. (703) 305-3230 


Authorized officer \ j \ 
Telephone No. /703^ 308-7728 ' 
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PCT/US99/24007 



I. Basis of the rcpurt 



1. With regard to the elenu'nts nl the intemaiional application:* 
[ X I the inlLTnalu)n^ll application as originally tiled 
the Ucscripiion. 



pages 
pages 
pages 



1-25 



NONE 



, as originally filed 

, filed with the demand 



NONE 



filed with the letter of 



the claims: 

pages 

pages 

pages 

pages 



26-33 



NONE 



NONE 



, as originally filed 

as amended (together with any statement) under Article 19 
, filed with the demand 



NONE 



filed with the letter of 



x] the drawings: 

pages 

pages 

pages 



1-15 



NONE 



, as originally filed 

, filed with the demand 



NONE 



the see|uencc listing ]Kiri of the description: 

pages NONE 

pages NONE 

pages NONE 



, filed with the letter of , 



, as originally filed 

, filed with the demand 



. filed with the letter of 



2. With regard to the language. ;ill the elements marked above were available or fiimished to this Authority in the language in wiiich 
the international application was tiled, unless otherwise indicated under this item. 

These elements were cnaiUiblc or furnished to this Authority in the following language which is: 

I I the language oi* a translation furnished for the purposes of international search (under Rule 23.1(b)). 
I I the language of publication of the international application (under Rule 48.3(b)). 

I I the language of the translation furnished for the purposes of international preliminary examination (under Rules 55.2 and/ 
or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the international 

preliminarv" examination v\as carried out on the basis of the sequence listing: 

□ contained in the inlcriutional application in printed form. 

[ I filed lo^elhcr \\nh the international application in computer readable form. 

I I furnished subsL'*.]ucnilv to this Authority in written form. 

I I furnislied subseL)iiLnil\ to this Authority in computer readable form. 

□ The statement thai ilic subseciuently furnished written sequence listing does not go beyond the disclosure in the 
intemaiional applicalioti as filed has been furnished. 

I I The statement tliai tlic ijilbrmation recorded in computer readable form is identical to the writen sequence listing has 
' — ' been furnished. 

4 I x| The amendments have resulted in the cancellation of: 

\—i the description, pages 

the claims. Nos. NONE 
\~x\ the drawings, sheets/ftft NONE 



5. I I This report has Ix-cn drawn as if (some ot) the amendments had not been made, since they have been considered to go 

beyond the disvjlusuiv as filed, as indicated in the Supplemental Box (Rule 70.2(c)).** 
* Replacement sfieeis wfii Ji have been furnished lo the receiving Office in respcjnse to an invitation under Article 14 are referred to 
in this reptnf as "uh'^itiaHy fifed'' and are nut annexed to this report since they do not coniain amendmerus (Rules 70.16 
and 70.17). 

**/4a?v replacement shti^i cioitaininfj such amendments must be referred to under item J and annexed to this report. 
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V. Reasoned statemcn. under Article 35(2) with regard to novelty, inventive step or industrial applicability 
citations and expla nations supporting such statement 

I. statemeni 

Novelty (N) 

Inventive Step (ISj 



Claims 1-60 



Claims none 



Claims 3-29, 31-58 



Claims 1-2, 30, 59-60 



YES 
NO 

YES 
NO 



Industrial Applicability (lA) Claims 1-60 ygg 

Claims none j^q 



2. citations and explanations (Rule 70.7) 
(See Supplemental Shcci.) 



Forai PCT/IPEA/409 (Box V) (July 1998) ★ 



INTERNATIONAL PRELIMINARY EXAMINATION REPORT 



I mernSiion 



TTona! application No. 
PCT/US99/24007 



Supplemental Box 

(To be used when the spcicc in any of the preceding boxes is not sutTicient) 



Continuation of: Boxes [ - Vfll Sheet 10 

CLASSIFICATION; 

The Imcrmtion:il P;iier:i Classification (IPC) and/or the National classification are as listed below 
IPC(7): G06K 9/00 ;nui US CI.: 382/128. 130. 131, 132. 133. 172, 209, 224. 256. 260. 266. 274. 275. 294 ; 378/62. 65. 
146; 600/300 . 407. 



V. 2. REASONED STATEMENTS - CITATIONS AND EXPLANATIONS (Continued): 

Claims 1-2. 30 and 59-60 lack ati inventive step under PCT Article 33(3) as being obvious over KanO et 31. (US. 

5,359,513) in view of Yamada et al. (US. 5,235,510). 

Regar(ding claims 1 and 2, Kano discloses a method and system for detection of 
interval change in temporally sequential chest images comprising the steps of : 

obtaining a digital first and a second image of a chest ( column 5, lines 60-65 
Figure 1C, step 10), 

performing image warping on one of the first and second images to produce a 
warped image which is registered to the other side of the first and second image ( 
column 5. lines 15-25, Figure IB, steps 44-46), 

subtracting the warped image from the other image to generate a subtraction 
image (column 5. lines 10-22, Figure IB, steps step 50), 

displaying the subtraction image ( Figures 2A-2D). 

Kano discloses the step of displaying the right and left lung areas (column 6, 
lines 35-45). However, Kano is silent about the specific details regarding producing' a 
second image which is a mirror image of the first image. 

However, in the same field of endeavor, however, Yamada discloses a computer- 
aided diagnostic system comprising the step of producing a second image which is 
mirror image of the first image (column 13, lines 20-30). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to use the steps of producing a mirror image as taught 
by Yamada in the system of Kano because Yamada provides Kano with a system which 
outputs computer-aided diagnosis data with a high precision by a simple operation 
having no adverse effect on diagnosis made by doctors. 

Regarding claim 30, Kano discloses the steps of detecting ribcage edges on both 
sides of the lungs ( column 6, lines 30-45) including the steps of fitting, shifting and 
rotating ( column 5, lines 1-55) to produce a lateral inclination corrected second image 
with the midline in the lateral inclination corrected image ( column 6, lines 12-44). 

Claims 59 and 60 are apparatus and computer program claims corresponding to 
claims 1-2 above and are similarly analyzed. 

Claims 3-29, 31-58 meet the criteria set out in PCT Article 33(2)-(4), because the prior 
art does not teach or fairly suggest the steps of : 

(a) performing lateral inclination correction on the original image to generate the first 
image, determining average horizontal locations of the left and right ribcage edges at 
plural vertical locations, as recited in claim 3; 

(b) adding a predetermined pixel value to the pixel values of the lung regions of the 
subtraction image; maintaining pixels of regions other than the lung regions of the 
subtraction image at one of: a constant pixel value to generate a uniform background 
and, at the values corresponding pixels in one of the first and second images; as 
recited in the claim 5; 
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(To be used when the sp;icc in any of the preceding boxes is not su fficient) 
Continuation of: Boxes I - VIII 

Sheet 1 1 

(c) removing from consideration false positive locations having an extracted feature 
which bears a predetermined relationship with respect to the predetermined threshold- 
as recited in the claim 9; 

(d) performing image warping on one of the second and fourth images to produce a 
warped image which is registered to the other of the said second and fourth imaaes- 

as recited in the claim 31; ' 

(e) performing surface fitting on the shift values of the right lung of the one image 
and independently performing surface fitting of shift values of the left lung of one image 
to smooth the shift values and generate fitted shift values for the left and right lungs of 
the one image; as recited in the claim 33. 



NEW CITATIONS 

NONE 
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Applicant's or agent's file reference 
0730006520WO 


FOR FURTHER see Notificatkm of Transnuttal of Internationa] Search Rqx>it 
ACTION (Form PCT/ISA/220) as well as, where applicable, item 5 below. 


International application No. 
PCT/US99/24007 


International filing date (day/month/year) 
05 NOVEMBER 1999 


(Earliest) Priority Date (ck^/month/year) 
OS NOVEMBER 1998 


Applicant 

ARCH DEVELOPMENT CORPORATION 



This international search report has been prepared by this International Searching Authority and is transmitted to the applicant 
according to Article 18. A copy is being transmitted to the International Bureau. 

This international search rqxirt consists of a total of L-^^T'^ sheets. 

I X| It is also accompanied by a copy of each prior art document cited in this report. 



1. [ j Certain claims were found unsearchable (See Box I). 

2. Unity of invention is tacking (See Box II). 

3. The international application contains disclosure of a nucleotide and/or amino acid sequence listing and the 
I— J international search was carried out on the basis of the sequence listing 

filed with the international application. 

furnished by the applicant separately from the international application, 

but not accompanied by a statement to the effect that it did not include matter 



□ 
□ 



□ 



□ 



going beyond the disclosure in the international application as filed, 
transcribed by this Authority. 



4. With regard to the title, [ | the text is approved as submitted by the applicant. 

j x| text has been established by this Authority to read as follows: 

SYSTEM FOR COMPUTERIZED I ROCESSING OF CHEST RADIOGRAPfflC IMAGES 



5. With regard to the abstract, 

I I the text is approved as submitted by the applicant. 

I x| the text has been established, according to Rule 38.2(b), by this Authority as it appears in 
Box III. The applicant may, within one month from the date of mailing of this international 
search report, submit comments to this Authority. 

6. The figure of the drawings to be published with the abstract is: 

Figure No. iA as suggested by the appUcant. Q None of the figures. 

I I because the applicant failed to suggest a figure. 

I I because this figure better characterizes the invention. 
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Box III TEXT OF THE ABSTRACT (Continuation of Item 5 of the first sheet) 



A method, system and computer readable medium for computerized processing of 
chest images including obtaining a digital first image of a chest (SI 00); producing a 
second image which is mirror image (S300) of the first image; performing image 
warping ( on one of the first and second images to produce a warped image ( S400) 
which is registered to the other of the first and second images subtracting the 
warped image from the other image to generate a substraction image (S600). 
Another embodiment includes obtaining a digital first image of a chest of a subject; 
detecting ribcage edges on both sides of the lungs in the first chest image; 
determining average horizontal locations of the left and right ribcage edges at 
plural vertical locations; fitting the determined average horizontal locations to 
straight line to derive a midline; rotating the chest image so that the mideline is 
vertical; and shifting the rotated image to produce a lateral inclination corrected 
(S200) second image with the midline centered in the laterla inclination corrected 
image. 
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(54) Tide: SYSTEM FOR COMPUTERIZED PROCESSING OF CHEST RADIOGRAPHIC IMAGES 
(57) Abstract 

A method, system and computer readable medium for computerized processing of chest 
images including obtaining a digital first image of a chest (SI 00); producing a second image 
which is a mirror image (S300) of the first image; performing image warping on one of the 
first and second images to produce a warped image (S400) which is registered to the other 
of the first and second images; and subtracting the warped image from the other image to 
generate a subtraction image (S600). Another embodiment includes obtaining a digital first 
image of the chest of a subject; detecting ribcage edges on both sides of the lungs in the first 
chest image; determining average horizontal locations of the left and right ribcage edges at 
plural vertical locations; fitting the determined average horizontal locations to a straight line 
to derive a midline; rotating the chest image so that the midline is vertical; and shifting the 
rotated image to produce a lateral inclination corrected (S200) second image with the midline 
centered in the lateral inclination corrected image. 
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SYSTEM FOR COMPUTERIZED PROCESSING OF CHEST RADIOGRAPHIC IMAGES 

STATEMENT REGARDING FEDERATJ.Y SPONSORED RFSFARPR 

The present invention was made in part with U.S. Government support under 
USPHS grant numbers CA62625 and CA64370 (National Institute of Health). The U.S. 
Government has certain rights in the invention. 

BACKGROUND OF THE INVENTION 

Field of th e Invention : 

The present invention relates generally to a computerized method and system 
provided to aid radiologists in detection of abnormalities, such as lung nodule, 
pneimiothorax, pneumonia, and bulla, in chest radiographs. 

The present invention also generally relates to computerized techniques for 
automated analysis of digital images, for example, as disclosed in one or more of U.S. 
Patents 4,839,807 ; 4,841,555; 4,851,984; 4,875,165; 4,907,156; 4,918,534; 5,072,384; 
5,133,020; 5,150,292; 5,224,177; 5.289,374; 5,319,549; 5.343,390; 5,359,513; 
5,452,367; 5,463,548; 5,491,627; 5,537,485; 5,598,481; 5,622,171; 5,638,458; 
5,657,362; 5,666,434; 5,673,332; 5,668,888; 5,740,268; 5,790,690; and 5,832,103; as 
well as U.S. patent applications 08/158,388 (PCT Publication WO 95/14431); 08/173,935; 
08/220,917 (PCT Publication WO 95/26682); 08/398,307 (PCT Publication WO 
96/27846); 08/523,210 (PCT Publication WO 95/15537); 08/536.149; 08/562,087; 
08/757,611; 08/758.438; 08/900.191; 08/900,361; 08/900,362; 08/900,188; 08/900,189, 
08/900,192; 08/979,623; 08/979,639; 08/982,282; 09/027,468; 09/027,685; 09/028,518; 
09/053,798; 09/092,004; 09/098,504; 09/121,719; 09/131,162; 09/141,535; and 
09/156,413, all of which are incorporated herein by reference. Of these patents and 
applications, 4,907,156; 5,072,384; 5,224,177; 5,289,374; 5,319,549; 5,359,513; 
5,463,548; 5,622,171; 5.790,690; 08/562,087; 08/562,188; 08/757.611; 08/758,438; 
08/900,191; 08/900,361; 08/900,362; 09/027,685; 09/053,789; and 09/121,719 are of 
particular interest. 

The present invention includes use of various technologies referenced and described 
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in the above-noted U.S. Patents and Applications, as well as described in the references 
identified in the appended APPENDIX by the author(s) and year of publication and cross- 
referenced throughout the specification by numerals in brackets corresponding to the 
respective references listed in the APPENDIX, the entire contents of which, including the 
related patents and applications listed above and references listed in the APPENDIX, are 
incorporated herein by reference. 

Discussion of the Background 

Detection of early lung cancers on chest radiographs is a difficult task for 
radiologists, because subde lesions tend to be low in contrast and can overlap with ribs and 
clavicles. To assist radiologists in the detection of newly developed abnormalities on chest 
radiographs, a temporal subtraction technique has been reported. [1] In this technique, a 
previous chest image is subtracted from a current chest image to produce a subtraction 
image. Subtle changes on chest radiographs can be enhanced on the subtraction image, and 
thus the detection accuracy of interval changes can be improved significantly by use of the 
temporal subtraction technique. [2] However, the temporal subtraction technique is not 
applicable in the absence of a previous chest radiograph. 

S UMMARY QF THE INVENTION 

Accordingly, it is an object of the present invention to provide a method and system 
for the computerized detection of asymmetric abnormalities in a single chest radiograph. 

It is another object of the present invention to provide a method and system for the 
computerized detection of asymmetric abnormalities in a single chest radiograph using a 
contralateral subtraction technique. 

It is another object of the present invention to provide a method and system for the 
computerized detection of asymmetric abnormalities in a single chest radiograph using a 
contralateral subtraction technique employing lateral inclination correction, a reversed 
"mirror" image, warping of the mirror image, and subtraction of the warped mirror image 
from the original image to derive the contralateral subtraction image. 

It is another object of the present invention to provide a method and system for the 
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computerized detection of asymmetric abnormalities in a single chest radiograph using a 
contralateral subtraction technique and successively applying three techniques to an initial 
contralateral subtraction image to acquire improved subtraction images. 

It is another object of the present invention to provide a method and system for the 
removal of false positives from difference images using a contralateral subtraction technique 
in computerized detection of lung nodules in single chest radiographs. 

It is another object of the present invention to provide a method and system for 
improved temporal subtraction using a midline detection method of a contralateral subtraction 
technique. 

These and other objects are achieved according to the present invention by providing a 
novel method, system and computer readable medium for computerized processing of chest 
images including obtaining a digital first image of a chest; producing a second image which 
is a mirror image of the first image; performing unage warping on one of the first and 
second images to produce a warped image which is registered to the other of the first and 
second images; and subtracting the warped image from the other image to generate a 
subtraction image. 

Another embodiment useful when temporally spaced images are available includes 
obtaining a digital first image of a chest of a subject; detecting ribcage edges on both sides of 
the lungs in the first chest image; determining average horizontal locations of the left and 
right ribcage edges at plural vertical locations; fitting the determined average horizontal 
locations to a straight line to derive a midline; rotating the chest image so that the midline is 
vertical; and shifting the rotated image to produce a lateral inclination corrected second image 
wdth the midline centered in the lateral inclination corrected image. 

The present invention also includes a computer readable medium storing program 
instructions by which the method of the invention can be performed when the stored program 
instructions are appropriately loaded into a computer, and a system for implementing the 
method of the invention. 

Thus, according to the present invention, a new contralateral subtraction technique 
based on one posteroanterior (PA) chest image has been developed. Since the rib structure 
is nearly symmetrical, the chest image of the right peripheral lung is generally similar to 
that of the left lung. Taking advantage of this, the technique includes lateral inclination 
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correction by rotating and shifting the original chest image so that the midline of the thorax 
is aligned with the vertical centerline of the original chest image, lateral reversal of the 
rotated image to produce a reversed "mirror" image, warping of the mirror image, and 
subtraction of the warped mirror image from the original image to derive the contralateral 
subtraction image. Thereafter, additional processing techniques can be successively applied 
to the initial contralateral subtraction image to acquire improved subtraction images. In the 
technique, a contralateral subtraction image can be obtained by subtracting the right/left 
reversed "mirror" image from a single chest image. Similar to the temporal subtraction 
technique, the contralateral subtraction scheme can cancel out most of symmetrical skeletal 
structures and enhance asymmetrical opacities, thus demonstrating subtle abnormalities 
more clearly. On the other hand, unlike the temporal subtraction technique, a subtraction 
image can be obtained whenever a single PA chest image is available. Therefore, the 
contralateral subtraction technique can be of significant clinical importance in some cases. 

BRIEF DESCRIPTION OF THE DRAWINGS 
A more complete appreciation of the invention and many of the attendant advantages 
thereof will be readily obtained as the same becomes better understood by reference to the 
following detailed description when considered in connection with the accompanying 
drawings wherein: 

Figure 1(a) is an overall scheme of a contralateral subtraction technique for a PA 
chest image; 

Figure 1(b) is an overall scheme for improved temporal subtraction using a midline 
detection method of a contralateral subtraction technique; 

Figure 1(c) is an overall scheme for the removal of false positives from difference 
images using a contralateral subtraction technique in computerized detection of lung nodules 
in single chest radiographs; 

Figures 2(a), 2(b), 2(c) and 2(d) are images showing for comparison (a) an original 
chest radiograph, (b) a contralateral subtraction image including all regions, (c) a 
contralateral subtraction image with a uniform background over an area outside the two 
lungs, and (d) a contralateral subtraction image with a "chest" background, which is the 
original image superimposed over the area outside the two lungs, respectively; 
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Figure 3 is an illustration of measures for errors of angle and location of a detected 
midline relative to a gold standard; 

Figure 4 is a chest radiograph with a gold standard (black solid line) for midline, 
detected midlines given by a profile based method (dashed line) and by a ribcage edge 
based method (dotted line); 

Figures 5(a), 5(b) and 5(c) are charts illustrating (a) distribution of errors for angles 
and locations from a gold standard for individual midlines indicated by three observers, (b) 
distribution of errors for a midline detected by a profile based method, (c) distribution of 
errors for a midline detected by a ribcage edge based method, respectively; 

Figures 6(a), 6(b) and 6(c) are images showing for comparison (a) an original chest 
radiograph, (b) a contralateral subtraction image obtained by using a profile based method 
for midline detection, and (c) an improved subtraction image obtained by using a ribcage 
edge based method for midline detection, respectively; 

Figures 7(a), 7(b) and 7(c) are images showing for comparison (a) an original chest 
radiograph, (b) a contralateral subtraction image obtained by including mediastinal and 
cardiac regions for shift value analysis, and (c) an improved subtraction image obtained by 
eliminating the mediastinal and cardiac regions from the shift value analysis, respectively; 

Figures 8(a), 8(b) and 8(c) are images showing for comparison (a) an original chest 
radiograph, (b) a contralateral subtraction unage obtained by using simultaneous fittings of 
shift values in right and left lungs, and (c) an improved subtraction image obtained by using 
separate fittings of shift values in the right and left lungs, respectively; 

Figures 9(a) and 9(b) are images showing for comparison (a) an indicated lung 
nodule, and (b) a contralateral subtraction image with an enhanced nodule, respectively; 

Figure 10 is a graph showing a distribution of contrasts of nodule candidates at 10 
mm and 20 mm diameters for nodules and non-nodules; 

Figure 1 1 is a graph showing a distribution of relative standard deviations for 
nodule candidates in object and background regions at 10 nmi diameter for nodules and 
non-nodules; 

Figure 12 is a graph showing a relationship between contrast and relative standard 
deviation at 10 mm diameter for nodules and non-nodules; and 

Figure 13 is a schematic illustration of a general purpose computer 100 programmed 
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according to the teachings of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The chest images used in development of the present invention consist of 50 normals 
and 50 abnormals with solitary lung nodules which were selected from 247 chest images in 
the Japanese Standard Digital Image Database developed by the Japanese Society of 
Radiological Technology. [3] The images were digitized with a 0.175 mm pixel size, a 
matrix size of 2048 x 2048 and 12 bits gray levels. However, the matrix size was reduced 
to 512 x 512 by subsampling of the original image data, and the number of gray levels was 
decreased to 10 bits. 

Referring now to the drawings, wherein like reference numerals designate identical 
or corresponding parts throughout the several views, and more particularly to Fig. 1(a) 
thereof, there is illustrated a top-level block diagram of a contralateral subtraction technique 
according to the present invention. 

First of all, for an obtained PA chest image in step SlOO, the lateral inclination 
which may be caused by an improper patient positioning is corrected by an image rotation 
technique at step S200. The rotated image is laterally (right/left) reversed to produce a 
reversed "mirror" image at step S300. The mirror image is registered to the original image 
by use of a cross correlation technique, and then deformed based on a nonlinear image 
warping technique in order to match peripheral ribs in the mirror image with those in the 
original image at step S400. [1] Finally, the warped mirror image is subtracted from the 
original image at step S500 to derive the contralateral subtraction image at step S600. The 
details of steps S100-S600 will now be described. 

Lateral inclination correction (Step S200) is designed particularly to correct lateral 
inclination of the thorax in the chest image. If the midline of the thorax is inclined slightly 
with an angle from the vertical direction, then the difference in the angles of the midlines 
between the original and the reversed chest images will become twice as large, which may 
lead to a serious misregistration error, and thus result in a poor subtraction image. It is 
therefore necessary to correct the lateral inclination before the subtraction technique is 
applied, by rotating the image so that the midline is in a vertical direction and then shifting 
the midline of the thorax to the vertical centerline of the original chest image, performed 
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Step S200. The rotating and shifting of the image can be, for example, achieved using 
conventional image rotation and shifting techniques as is known in the imaging arts. 

Midline detection: 

In the past, a profile based midline detection method has been applied to several 
computer-aided diagnosis (CAD) schemes with relatively satisfactory results. [5, 6] First, 
nine regions of interest (ROIs) are selected in the mediastinal region from the top to the 
bottom of the lung. To reduce the effect of noise level on the determination of the midline, 
the pixel values in each ROI are averaged along the vertical direction, which gives rise to a 
one dimensional (1-D) horizontal profile. For each profile, the maximum pixel value and 
its corresponding position are determined. Finally, the midline is determined by fitting the 
nine points to a straight line. 

If the pixels along the midline have local maximum values along a straight line, then 
this method can give satisfactory results. However, in many cases, the pixels on the 
midline are not necessarily local maximum values. For example, the pixels in the cardiac 
region are often greater than those near the midline, and thus the detected midline is 
commonly shifted to the cardiac region, resulting in an angulated incorrect midline, as will 
be illustrated later. Moreover, even though the pixels on the midline have local maximum 
values, the difference between these values and pixel values in the adjacent area is usually 
very small, which makes the prior method intrinsically vulnerable to image noise. 
Therefore, according to the present invention, a new ribcage edge based midline detection 
method has been devised, next described. 

First, the ribcage edges on both sides of the lungs are detected. Second, the average 
horizontal locations of the left and right ribcage edges are determined and then fitted to a 
straight line to derive the midline. The ribcage edges are detected by analyzing the first 
and second derivatives of profiles through chest images. [4] The detected ribcage edges are 
then fitted to a third order polynomial function to form smooth curves and also reduce 
noise, as described in detail elsewhere. [4] The ribcage edge detection method and the 
previous midline detection method described above, both detect edges or peaks by 
analyzing profiles and fitting them to some functions. However, the results of ribcage edge 
detection are far more reliable than the results of the previous midline detection because the 
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contrast on the ribcage edges is usually much greater. This is why the present invention 
preferably employs the ribcage edges for determination of the midline. 

Once the right and left ribcage edges are obtained, the average horizontal locations 
(i.e., mid-points) of the right and left ribcage edges at the same vertical position of the 
chest image are then determined. Due to the nearly synunetrical property of the ribcage 
edges on both sides of the lungs, most of the mid-points (or average locations) lie on a 
straight line, and thus these points are fitted to a straight line to produce the expected 
midline. In other words, a straight line to divide the lungs into two approximately 
symmetrical and equal parts is determined. Since the mid-points are derived from the 
average horizontal locations of the right and left ribcage edges, the variance of errors in 
these mid-points is expected to be a half of that in the detected ribcage edges, which would 
also contribute to a higher accuracy of the detected midline. 

Determination of gol d standard for the midline of chest imag es: 

In order to evaluate the accuracy of the detected midline, "truth" data about the 
midline on PA chest images are required. According to the present invention, a "gold 
standard" based on subjective judgments was obtained. First of all, it is assumed that a 
midline can be represented by a straight line. For each of 100 chest images, three 
observers independently indicated both ends of a midline on the computer screen by using a 
mouse. The corresponding end points of three located midline are averaged to yield the 
gold standard for each case. 

Nonlinear warping of right/left reversed "mirror" imape fSteps S300. S40QV 

At step S300, a right/left reversed mirror image is obtained by laterally reversing 
the original image after the lateral inclination is corrected at step S200. This reversed 
mirror image will be warped and then subtracted from the original image to produce the 
contralateral subtraction image. A nonlinear image warping technique has been used 
successftilly in the temporal subtraction technique applied to chest images as described in 
detail previously (see, e.g., Patent application serial Number 09/053,789). According to 
the present invention, a similar image warping technique to obtain image registration is 
employed. However, proper modifications are made in image warping as applied to 
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contralateral subtraction as described below. The nonlinear image warping technique 
includes initial global matching, detailed local matching of peripheral ribs, determination of 
shift values, and coordinate conversion. In addition, nonlinear image warping can be 
performed iteratively to yield improved results. [5] 

Initial global matching on overall chest image: 

The initial global matching technique, which has been already applied to the 
temporal subtraction technique, is used for aligning the approximate lung area in the 
reversed mirror image to that in the original image. [6] First, the matrix sizes of two 
images, i.e., the original chest image and the reversed mirror image are reduced by a factor 
of four, and smoothed by a Gaussian filter to reduce the effect of the fine anatomical 
structures, such as small vessels, bronchi, devices and catheters on global matching of the 
two images. The lungs are then extracted from each of the two images by using ribcage 
edge information described above, and the region outside the ribcage edges is ignored. 
Finally, the upper parts of the lungs are aligned by using a cross correlation technique. 

Local image matching of peripheral ribs exclu d ing mediastinal and cardiac regions 

To perform local image matching of peripheral ribs in the right and left lungs, it is 
necessary to select a large number of template ROIs and search area ROIs. The template 
ROIs and the corresponding search area ROIs are automatically located within the lung 
regions of the original image and the reversed image, respectively, in the same way as 
those ROIs were located by the temporal subtraction technique. [1] The matrix sizes of 
template and search area ROIs are 32 x 32 and 64 x 64, respectively. It is important to 
note that with the contralateral subtraction technique, the mediastinal and cardiac regions 
are excluded for the selection of the ROIs, because the ROIs in the mediastinal and cardiac 
regions do not contain useful information for reliable image matching for the contralateral 
subtraction. [7] Once the template ROIs and search area ROIs are determined, a cross 
correlation technique is employed to perform the local image matching for the 
determination of the shift values ax and Ay in two orthogonal directions, which indicate a 
shift of the coordinates of the center of a search area ROI in the reversed mirror image, for 
the best match of the template ROI with the corresponding "matched" area in the search 
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area ROI. [1] 

Separate fittings of sh ift values in the right and left lungs : 

Since there are some variations in the shift values over the lung areas, a two 
dimensional surface fitting using a 10th order polynomial is applied to smooth each set (ax 
and Ay) of the shift values. [1] Initially, each set of the shift values in both the right and 
left lungs was fitted simultaneously, as was applied for the temporal subtraction technique. 
However, with the contralateral subtraction it was noted that the shift values in the right 
and left lungs can be quite different in some cases. Therefore, the shift values in the right 
and left lungs are fitted separately and independently for the contralateral subtraction. This 
separate fitting technique is possible because it is not necessary in contralateral subtraction 
to have one smooth function over the entire chest image. The four sets of fitted 10th order 
polynomials for each of ax and Ay are then used to perform the coordinate transformation 
on each of the right and left lungs independently in the reversed image, which produces a 
warped reversed image for the contralateral subtraction. 

Display of contralateral subtraction images (Steps S500. S600V 

A contralateral subtraction image is obtained by subtracting the warped reversed 
image from the original image at step S500. At step S600, in order to enhance image 
contrast to a proper level, the pixel values of the subtraction image may be multiplied by a 
factor such as 1.5 and 2.0. However, a factor of 1.0 can be used also for maintaining the 
same contrast of abnormal opacities in the contralateral subtraction image as that in the 
original image. A constant pixel value of 512 is then added to the contrast enhanced 
subtraction image, and the background region outside lungs is assigned a constant pixel 
value of 512 in order to display only the contralateral subtraction image of peripheral lungs. 
Note that the display of the mediastinal and cardiac regions is not useftil and can be 
disturbing to radiologists due to a large "misregistration" artifacts. Fig. 2(a) shows an 
original chest image, and Figs. 2(b) and 2(c) show, for comparison, contralateral 
subtraction images without and with the uniform background, respectively. The subtraction 
image with the uniform background can present effectively the asymmetric opacities in lung 
regions by dark patterns in the ipsilateral side and light patterns in the contralateral side. 
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Note also that the original PA chest image outside lung areas, instead of the constant 
background, can be displayed to maintain the general appearance of a "chest" background, 
as shown in Fig. 2(d). In fact, many radiologists preferred the display of the Fig. 2(d) 
contralateral subtraction image. 

Subjective evaluation for the qual ity of subtraction image: 

To evaluate subjectively the quality of the contralateral subtraction image, a five- 
point rating score, which has been used previously for subjective evaluation of the temporal 
subtraction image, is employed according to the present invention, that is, 
5 (excellent): all ribs are perfectly registered, (and thus eliminated), 
4 (good): most ribs are almost completely registered with very minor 
misregistration errors partly, 

3 (adequate): most ribs are well registered with some minor misregistration errors, 
2 (poor): most ribs are not well registered and appear in half of the intercostal 
space, and 

1 (very poor): most ribs are not registered and appear in the entire intercostal 

space. 

Another rating method is also used to examine the change in the quality of the 
subtraction image due to the use of an improved technique which will be later described. 
The rating score ranges from -2 to +2 as follows, when the quality of the subtraction image 
is 

+2: clearly improved, 
4- 1 : moderately improved, 
0: unchanged, 

-1: moderately degraded, and 
-2: clearly degraded. 

Comparison of accur acv for two midline detection methods : 

The accuracy of the detected midline is evaluated by determining the errors of angle 
and location of the midline compared with the gold standard. Fig. 3 shows the definitions 
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for the errors of the angle and the location of the detected midline. The error of angle 
between the detected midline and the gold standard is defined by the angle between the 
orientations of the detected midline and the gold standard. The error of location is defined 
by the horizontal distance of the detected midline from the gold standard at the vertical 
position of the midpoint between the two ends of the gold standard. 

Fig. 4 shows a chest radiograph with the gold standard (solid line) and midlines 
detected by the profile based method (dashed line) and the ribcage edge based method 
(dotted line). The detected ribcage edges are also shown by the solid curves. The midline 
by the ribcage edge based method is very similar to the gold standard, whereas the midline 
by the profile based method is angulated and quite different from the gold standard 
probably due to the effect of the cardiac region, as described earlier. 

Fig. 5(a) shows the distribution of the errors of angles and locations from the gold 
standard for individual midlines indicated by three observers. It is apparent that most 
points are distributed around the origin, which indicates that the errors are relatively small 
and the midline data given by three observers are quite consistent. Figures 5(b) and 5(c) 
show the distribution of the errors for the detected midlines given by the profile based and 
the ribcage edge based methods, respectively. It is clear that the ribcage edge based 
method can provide more accurate results than the profile based method. Table 1 shows the 
comparison of the standard deviations for errors of angles and locations for the midlines 
measured by observers and by the two midline detection methods. Note that the standard 
deviations obtained by the ribcage edge based method are considerably smaller than those 
by the profile based method, and comparable to those by observers. 

Table 1 . RMS errors from gold standard for the angles and locations of the midlines 



measured by various methods 





RMS errors of angles 
(degree) 


RMS errors of locations 
(mm) 


Observers* data 


0.56 


1.2 


Profile based midline detection method 


2.2 


9.6 


Ribcage edge based midline detection 
method 


0.81 


1.8 



-12- 



wo 00/28466 



PCTAJS99/24007 



Improvements of contralateral subtraction ima ges hv three techniques : 

Table 2 demonstrates the distribution of the number of cases for different 
subjective rating scores on the quality of subtraction images, which are obtained with an 
initial scheme and also by incorporating successively the three techniques. 

Table 2. Distribution of the numbers of chest images for different subjective rating scores 
on the quality of contralateral subtraction images, which are obtained with an 



initial scheme and by incorporating additional techniques for improvements 



Rating 


(a) 


(b) 


(c) 


(d) 


score* 


initial 


plus lateral inclination 


plus exclusion of 


plus separate 




technique 


correction 


cardiac region 


fittings on two 
lungs 


5 


3 


7 


12 


15 


4 


35 


33 


35 


36 


3 


35 


48 


42 


40 


2 


22 


11 


10 


8 


1 


5 


1 


1 


1 



* 5: excellent, 3: adequate, I: very poor. 



Table 3 shows the distribution of the number of cases for the change in subjective 
rating scores on the quality of subtraction images due to the successive use of the three 
techniques. At the initial phase of this study, 100 cases of chest radiographs with the initial 
contralateral subtraction scheme without the three techniques were examined. 

Table 3. Distribution of the numbers of chest images affected by three techniques in terms 
of subjective rating scores on the quality of contralateral subtraction images 
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Rating score* 


(a) 


(b) 


(c) 




lateral inclination 


exclusion of cardiac 


separate fittings on 




correction 


region 


two lungs 


+2 


9 


2 


0 


+ 1 


18 


16 


10 


0 


62 


75 


89 


-1 


11 


7 


1 


-2 


0 


0 


0 



+2: clearly improved, 0: no change, -2: clearly degraded. 



As shown in Table 2, 73 cases (73%) of 100 subtraction radiographs showed 
adequate, good, or excellent quality, whereas the remaining 27 cases were severely 
misregistered. Many of these misregistered cases can be improved by incorporating the 
three techniques, as described below. 

The first technique is the use of an improved midline detection method based on 
the ribcage edges, instead of the profile based method, to correct for the lateral inclination 
of the thorax in chest radiographs. As shown in the first column (a) of Table 3, this 
technique can improve the contralateral subtraction images considerably, especially for 
those cases with initially lower subjective rating scores. This can also be observed from the 
second column (b) of Table 2, where the numbers of cases with rating scores of 1 and 2 are 
decreased considerably after the new midline detection method is used. Fig, 6 shows 
comparison of (a) an original chest radiograph, (b) a contralateral subtraction image 
obtained by using a profile based method for midline detection, and (c) an improved 
subtraction image obtained by using a ribcage edge based method for midline detection. It 
is apparent in Fig. 6 that the subtraction image is clearly improved by using the new 
midline detection method. 

The second technique is the elimination of the mediastinal and cardiac region for 
the shift value analysis. This technique is equivalent to the application of the contralateral 
subtraction technique to lung areas alone. The improvement in the contralateral subtraction 
images is indicated in the second column (b) in Table 3, and the rating scores for the 

-14- 



wo 00/28466 




improved quality of subtraction images are listed in the third column (c) in Table 2. Fig. 7 
shows comparison of (a) an original chest radiograph, (b) a contralateral subtraction image 
obtained by including mediastinal and cardiac regions for shift value analysis, and (c) an 
improved subtraction image obtained by eliminating the mediastinal and cardiac regions 
from the shift value analysis. It is apparent in Fig. 7 that the subtraction images are ftirther 
improved by adding the second technique. 

The third technique is the separate fittings of shift values on the right and left 
lungs independently. The last columns in Tables 2 and 3 show the improvement made by 
this technique and the final rating scores for the quality of subtraction image, respectively. 
By employing the three techniques, ninety one (91 %) of 100 cases are rated as score 3 or 
above. Figure 8 shows comparison of (a) an original chest radiograph, (b) a contralateral 
subtraction image obtained by using simultaneous fittings of shift values in right and left 
lungs, and (c) an improved subtraction image obtained by using separate fittings of shift 
values in the right and left lungs. These experimental results indicate that the contralateral 
subtraction technique can provide good subtraction images for most of cases. 

The contralateral subtraction technique can be used to detect asymmetric 
abnormalities, for example, lung nodule, pneumothorax, pneumonia, and bullae, on single 
chest radiograph. Fig. 9(a) shows a chest radiograph with a lung nodule indicated by an 
arrow, and its contralateral subtraction image in Fig. 9(b). It is apparent that the lung 
nodule is enhanced in the subtraction image, since most of symmetric skeletal structures are 
canceled out. 

According to the present invention, it was assumed that the midline of the thorax 
can be represented approximately by a straight line. It is believed that this assumption is 
valid in most of PA chest images. However, due to large variations of individual 
radiographs, this assumption is not always valid to describe midlines in all clinical chest 
radiographs. In some cases, a nonlinear model to represent the midlines adequately may be 
needed. For example, a polynomial with the order of 2 or 3 may be suitable for 
representing the curved midline. Nevertheless, according to the present invention the linear 
model was used, because the midline is used only for correcting the lateral inclination of 
the thorax in a chest radiograph. Note that the linear model is simple and adequate for this 
invention. A complex nonlinear model may complicate the problem and lead to difficulty 
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in determining a proper correction of the midline. 

In the past, the subtraction images have been displayed with the small matrix size 
of512x512, because the cross-correlation technique and the subsequent image warping 
technique have been applied to this small matrix size. According to the present invention, a 
technique to display a high quality, large matrix subtraction image, and equivalently a high 
quality subtraction image with a small pixel size was devised. This is accomplished by 
employing a large matrix image such as 1024 x 1024 or 2048 x 2048 for image warping of 
the reversed mirror image for the contralateral subtraction technique and the previous 
image in the case of the temporal subtraction technique, as will be further discussed. 
Although the shift values of the coordinate transformation are determined initially on the 
512 X 512 matrix image, the corresponding shift values for the large matrix size are 
obtained by scaling using a factor of two or four for the matrix size of 1024 x 1024 or 2048 
X 2048, respectively, of the reversed mirror image (or the previous image in the case of 
temporal subtraction technique). The large matrix subtraction image is obtained by 
subtracting the large matrix warped image from the large matrix original chest image. This 
technique for display of the larger matrix size and thus the use of a small pixel size was 
very useftil for producing high quality subtraction images by elimination of pixel edges and 
pixel artifacts due to the use of a small matrix size (or a large pixel size) for the temporal 
and the contralateral subtraction techniques. It is useful also to apply an image processing 
technique such as a median filter and a Gaussian filter for smoothing the subtraction images 
by elimination of image noise and relatively small misregistration artifacts. 

Improved temporal subtraction t echnique using the new midline detection method : 

The above-described three techniques to improve contralateral subtraction images 
according to the present invention are applicable to the temporal subtraction technique [1,2] 
for improvement of the quality of temporal subtraction images. The basic scheme of the 
temporal subtraction technique is briefly described here. A number of template ROIs (32 x 
32 matrix) and the corresponding search area ROIs (64 x 64 matrix) are selected from the 
previous and the current images, respectively, to perform local matching of image details in 
the two images. The shift values, ax and Ay, for all pairs of selected ROIs are then 
determined by using a cross-correlation technique to find the best matched areas in the 
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search area ROL A two-dimensional surface fitting using a polynomial function is applied 
to each set of mapped shift values, ax and Ay, for conversion of the x, y coordinates of the 
previous image, i.e., for warping of the image. The warped previous image is then 
subtracted from the current image to produce the temporal subtraction image. The 
temporal subtraction image has been shown to be very useful to detect subtle interval 
changes in chest images. In order to reduce misregistration artifacts in the temporal 
subtraction image, a second image warping technique can be employed by using the current 
and the first warped previous image. The final temporal subtraction image is obtained by 
subtraction of the second warped previous image from the current chest image. The image 
warping technique can be applied repeatedly and thus iteratively; this technique is therefore 
called an iterative image warping technique. [5] 

Fig. 1(b) is an overall scheme for improved temporal subtraction using the 
previously described midline detection method of the contralateral subtraction technique (e.g., 
Fig. 1(a) step S200). In Fig. 1(b), at step SlOO* current and previous chest images are 
obtained. In step S200', lateral inclination correction is performed on the obtained images 
using the midline detection method of the contralateral subtraction technique to reduce effects 
of misregistration. Next, at step S500', temporal subtraction is performed and at step S600' 
the temporally subtracted image is displayed. 

According to the present invention, it was found that the ribcage edge based 
technique for an improved midline detection was particularly useful, as the results of 
subjective evaluation are illustrated in Tables 4 and 5. 

Table 4. Distribution of the numbers of chest images for different subjective rating scores 
on the quality of temporal subtraction images, which are obtained by using 
profile based and ribcage edge based midline detection method to correct lateral 
inclination 



Rating score* 


Previous method by profile 
based midline detection 


New method by ribcage edge 
based midline detection 


5 


22 (12.2%) 


27 (14.9%) 


4 


82 (45.3%) 


88 (48.6%) 


3 


65 (35.9%) 


55 (30.4%) 
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2 


10 (5.5%) 


8 (4.4%) 


1 


2 (1.1%) 


3 (1.7%) 



5: excellent, 3: adequate, 1: very poor. 



Table 5. Distribution of the numbers of chest images for different subject rating scores on 
the improvement of quality of temporal subtraction images by replacing profile 



Rating score* 




+2 


13 (7.2%) 


+ 1 


45 (24.9%) 


0 


107 (59.0%) 


-1 


13 (7.2%) 


-2 


3 (1.7%) 



+2: clearly improved, 0: no change. -2: clearly degraded. 



According to the present invention, 181 pairs of current and previous chest images, 
which were obtained from the lung cancer screening program at Iwate prefecture in Japan 
were employed. These chest images are obtained by using a Fuji computed radiography 
(CR) system which was installed in a mobile screen unit. Digital image data from the CR 
system are directly used for calculating the temporal subtraction images using the image 
warping technique. It is apparent in Tables 4 and 5 that the quality of temporal subtraction 
images is considerably improved by using the new midline detection technique. 

The technique to display a high quality, large matrix subtraction image, as described 
above, can be applied to temporal subtraction images. 

Contralateral subtraction: Removal of false po s itives in computerized detection of hmg 
nodules in chest radiographs; ■ 

The contralateral subtraction technique can be incorporated into a computer-aided 
diagnosis scheme to improve the overall performance. For example, it is very difficult to 
eliminate some false positives caused by rib-rib crossings or vessel-rib crossings in the 
computerized detection of lung nodules in chest radiographs using a difference imaging 
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scheme. These false positives can be eliminated by analyzing image features derived from 
the contralateral subtraction image. 

The computerized difference imaging scheme is briefly described here. [8] First, 
digital chest images or a digitized version of conventional chest radiographs are subjected to 
two filtering operations which are operated in parallel. One produces an enhanced image of 
nodules by using a filter such as a matched filter, and the other provides a suppressed 
image of nodules by using another filter such as a ring average filter. A difference image is 
then obtained by subtraction of the suppressed image from the enhanced image. The 
difference image contains strongly enhanced nodules and strongly suppressed background, 
which is due to the removal of the majority of the normal anatomic background structures, 
and therefore is useful for identifying candidates of nodules . This difference image 
technique is considered a generalization of edge enhancement techniques, and in fact the 
difference image looks very similar to edge-enhanced images. If the two filters are linear 
filters, then the two can be combined into one filter operation. However, if a nonlinear 
filter is applied, two filtering operations need to be applied separately in parallel. The 
difference image technique is applicable to computerized detection of isolated abnormal 
patterns such as microcalcifications and masses in manunograms. 

The initial candidates of lung nodules are identified from relatively round patterns 
with large pixel values in the difference image, which may include nodules as well as 
normal structures such as ribs and pulmonary vessels. Therefore, image features of these 
candidates are extracted and quantified in terms of the size, contrast, and other parameters 
related to the shape of the candidates. Specifically, the extracted features are related to 
gray level, morphology, or edge gradient, such as effective diameter, degrees of circularity 
and irregularity, slopes of the effective diameter and degrees of circularity and irregularity, 
average gradient, standard deviation of gradient orientation, contrast and net contrast (e.g., 
as taught in Patent Application No. 08/562,087). A rule-based method (e.g., as taught in 
Patent Nos. 5,463,548 and 5,622,171 and Patent Application Nos. 08/562,087; 
08/562,188; 08/758,438; 08/900,361; and 09/027,685) is then applied which removes some 
candidates as false positives, when their features are matched to those of normal anatomic 
structures such as end-on vessels, rib-rib crossings, rib- vessel crossings, aggregates of 
vessels, and rib-clavicle crossings. 
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Finally, a trained artificial neural network (ANN), trained by use of a number of 
image features, is employed for further removal of false positives in the remaining 
candidates (e.g., as taught in Patent Nos. 5,463,548 and 5,622,171 and Patent Application 
Nos. 08/562,087; 08/562,188; 08/758,438; 08/900,361; and 09/027,685). When the database 
of 100 chest images with lung nodules and 100 normal cases was used, the performance of 
the difference imaging scheme provided 70% sensitivity at a false positive rate of 1.7 per 
image. 

Figure 1(c) is an overall scheme for the removal of false positives from difference 
images using a contralateral subtraction technique in computerized detection of limg nodules 
in single chest radiographs. 

According to the present invention, 10 chest radiographs, each of which included 
solitary lung nodules were obtained (step SI GO). Results of this study indicated that all of 
ten nodules were correctly detected by the computerized difference imaging scheme, but the 
total of 14 false positives appeared in lung fields of ten chest images were reported (steps 
S700-S800). All of these false positives were related to ribs such as rib-rib crossings, rib- 
vessel crossings and rib-clavicle crossings. 

Then the contralateral subtraction images were obtained for all of ten chest images 
(steps S200-S600). All of the 24 locations on original chest images, which included 10 
nodules and 14 false positives, indicating the computer output of the nodule detection 
scheme were then converted to the corresponding locations on the contralateral subtraction 
images (step S900). This conversion was necessary because the contralateral subtraction 
images were obtained after the original chest images were rotated and shifted for correction 
of the lateral inclination (steps S200-S600), as described earlier. 

In contralateral subtraction images, when ribs in one lung match with those in 
another lung, ribs are generally eliminated and thus have uniform density which is the same 
as the adjacent background, and nodules appear as dark round patterns. Therefore, false 
positive computer output due to ribs can be identified, if image features associated with a 
computer detected candidate would match those of ribs. The contrast of the candidate as an 
image feature to distinguish between nodules and false positives due to ribs was employed 
(step SI 000). The contrast is defined here as the difference in the average pixel values 
between the central area of the candidate and the immediately adjacent background area. 
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The average pixel value in the central area of the candidate was empirically determined 
over a circle with a diameter d, whereas the average pixel value in the background area was 
determined over a doughnut shape area with the inner diameter of d and the outer diameter 
of 2d. The contrasts of all candidates including nodules and non-nodules (false positives) 
were determined, for example, over a wide range of the diameter d from 2 mm to 40 mm. 
It was found that the contrasts of nodules in contralateral subtraction images tend to be 
generally greater than those of non- nodules. For example, Fig. 10 shows that the contrasts 
of nodules at 10 nun and 20 mm diameters are greater than 20 pixel values (dotted line), 
whereas there are six non-nodules with low contrast below 20 pixel values. Therefore, this 
result indicates that six false positives can be removed from the list of the computer output 
indicating the potential sites of lung nodules, if the nodule candidates in contralateral 
subtraction images, which contain the contrast less than a predetermined threshold level 
such as 20 pixel values, would be eliminated. 

However, there are eight non-nodules with large contrasts which are comparable to 
the contrasts of ten nodules. It was found that the large contrast for some non-nodules was 
caused by misregistration artifacts in contralateral subtraction images. Since the patterns 
due to misregistration artifacts generally include localized mixture of light and dark 
densities, it is expected that the standard deviation of pixel values in the central area and/or 
the background area would be large compared to those areas without misregistration 
artifacts. Fig. 11 shows the relative standard deviations in the central and the background 
area. The relative standard deviation was determined by the ratio of the standard deviation 
of pixel value variation to the average pixel value in the central area and/or the background 
area. It is apparent in Fig. 11 that the relative standard deviation for nodules are relatively 
small, but there are five non-nodules with large relative standard deviations above 15% 
(dotted line), which were due to misregistration artifacts. Therefore, this result illustrates 
that five false positives can be removed if the relative standard deviation of a nodule 
candidate in contralateral subtraction image would have a large value above a 
predetermined threshold level (step SHOO). By combining the results in Figs. 10 and 11, 
ten out of 14 false positives can be eliminated, since there is one "overlapped" false positive 
which can be removed either in Fig. 10 or Fig. 11. 

Another method for elimination of false positives is illustrated in Fig. 12, where the 
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correlation between the contrast and the relative standard deviation of the central areas of 
the nodules is demonstrated. It is apparent in Fig. 12 that nodules tend to have large 
contrasts and small standard deviations, whereas non-nodules tend to have small contrasts 
or large contrasts with large relative standard deviations. Therefore, when a contrast 
versus standard deviation threshold as indicated by the dotted line is employed, 13 out of 14 
false positives can be eliminated (step S1200-S1300), This result indicates that a large 
number of false positives can be eluninated and thus can improve substantially the 
performance of the computerized scheme for detection of lung nodules in chest images by 
employing contralateral subtraction images. 

Fig. 13 is a schematic illustration of a computer system for detection of asymmetric 
abnormalities in a single chest radiograph. A computer 100 implements the method of the 
present invention, wherein the computer includes, for example, a display device 102, such as 
a touch screen monitor with a touch-screen interface, a keyboard 104, a pointing device 106, a 
mouse pad or digitizing pad 108, a hard disk 1 10, or other fixed, high density media drives, 
coimected using an appropriate device bus (e.g., a SCSI bus, an Enhanced IDE bus, an 
Ultra DMA bus, a PCI bus, etc.), a floppy drive 1 12, a tape or CD ROM drive 1 14 with tape 
or CD media 1 16, or other removable media devices, such as magneto-optical media, etc., and 
a mother board 118. The mother board 1 1 8 includes, for example, a processor 120, a RAM 
122, and a ROM 124 (e.g., DRAM, ROM, EPROM, EEPROM, SRAM, SDRAM, and 
Flash RAM, etc.), I/O ports 126 which may be used to couple to an image acquisition device 
(not shown), and optional special purpose logic devices (e.g., ASICs) or configurable logic 
devices (e.g., GAL and reprogrammable FPGA) 128 for performing specialized 
hsirdware/software functions, such as sound processing, image processing, signal processing, 
neural network processing, etc., a microphone 130, and a speaker or speakers 132. 

As stated above, the system includes at least one computer readable medium. 
Examples of computer readable media are compact discs, hard disks, floppy disks, tape, 
magneto-optical disks, PROMs (EPROM, EEPROM, Flash EPROM), DRAM, SRAM, 
SDRAM, etc. Stored on any one or on a combination of computer readable media, the 
present invention includes software for controlling both the hardware of the computer 100 
and for enabling the computer ICQ to interact with a human user. Such software may 
include, but is not limited to, device drivers, operating systems and user applications, such 
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as development tools. Such computer readable media further includes the computer 
program product of the present invention for performing any of the processes according to 
the present invention, described above (see, e.g.. Figs. l(a)-(c)). The computer code 
devices of the present invention can be any interpreted or executable code mechanism, 
including but not limited to scripts, interpreters, dynamic link libraries, Java classes, and 
complete executable programs, etc. 

The programming of general purpose computer 1 00 may include a software module 
for digitizing and storing images obtained from an image acquisition device (not shown). 
Alternatively, the present invention can also be implemented to process digital data derived 
from images obtained by other means, such as a picture archive communication system 
(PACS). In other words, often the digital images being processed will be in existence in 
digital form and need not be converted to digital form in practicing the invention. 

The invention may also be implemented by the preparation of application specific 
integrated circuits or by interconnecting an appropriate network of conventional component 
circuits, as will be readily apparent to those skilled in the art. 

In addition, although the present invention is defined in terms of eliminating false 
positives in candidate nodules, based on contrast, relative standard deviation, and/or 
correlation between the contrast and the relative standard deviation of the central and/or 
background areas of the nodules, other image features and/or other areas of the nodules can 
be used, as will be readily apparent to those skilled in the art. 

Recapitulating, according to the present invention, a novel contralateral subtraction 
technique for the detection of asymmetric opacities on single chest radiograph has been 
developed, and the three techniques (i.e., lateral inclination correction, exclusion of cardiac 
region, and separate fitting of shift values for left and right lungs) to improve the 
contralateral subtraction image have been applied. The contralateral subtraction technique 
can remove most of peripheral ribs, and thus enhance low-contrast peripheral lesions on 
chest radiographs. It is believed that this technique can assist radiologists in the detection 
of subtle lung opacities. 

Obviously, numerous modifications and variations of the present invention are 
possible in light of the above teachings. It is therefore to be understood that within the 
scope of the appended claims, the invention may be practiced otherwise than as specifically 
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described herein. 
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Claims: 

1. In a method of computerized processing of chest images, the improvement 
comprising: 

obtaining a digital first image of a chest; 

producing a second image which is a mirror image of the first image; 
performing image warping on one of the first and second images to produce a warped 
image which is registered to the other of said first and second images; and 

subtracting the warped image from the other image to generate a subtraction image. 

2. The method of Claim 1, further comprising: 
displaying the subtraction image. 

3. The method of Claim 1, wherein said step of obtaining a digital first image 
comprises: 

obtaining a digital original image; and 

performing lateral inclination correction on the original image to generate said first 

image, comprising, 

detecting ribcage edges on both sides of the lungs in the original image, 
determining average horizontal locations of the left and right ribcage edges at plural 

venical locations, 

fitting the determined average horizontal locations to a straight line to derive a midline, 
rotating the original image so that the midline is vertical, and 

shifting the rotated image to produce said first image with the midline centered in said 
first image. 

4. The method of Claim 3, wherein said step of performing image warping comprises: 
performing global matching to align the lung areas of the one image to the other image; 
performing local matching of selected regions of interest in the right and left lungs of said 

one image to the right and left lungs of the other image to generate shift values for pixels of said 
one image; 

performing surface fitting on the shift values of the right lung of the one image and 
independently performing surface fitting of shift values of the left lung of the one image to 
smooth the shift values and generate fitted shift values for the left and right lungs of the one 
image; 
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shifting pixels of the one image using the fitted shift values. 

5. The method of Claim 1 , further comprising: 
producing an enhanced subtraction image, including: 
enhancing contrast of the lung regions of the subtraction image; 

adding a predetermined pixel value to the pixel values of the lung regions of the 
subtraction image; 

maintaining pixels of regions other than the lung regions of the subtraction image at one 

of, 

(1) a constant pixel value to generate a uniform background, and 

(2) at the values of corresponding pixels in one of the first and second images; and 
displaying the enhanced subtraction image. 

6. The method of Claim 4, ftxrther comprising: 
producing an enhanced subtraction image, including: 
enhancing contrast of the lung regions of the subtraction image; 

adding a predetermined pixel value to the pixel values of the lung regions of the 
subtraction image; 

maintaining pixels of regions other than the lung regions of the subtraction image at one 

of, 

(1) a constant pixel value to generate a uniform background, and 

(2) at the values of corresponding pixels in one of the first and second images; and 
displaying the enhanced subtraction image, 

7. The method of Claim 4, wherein the step of performing image warping comprises: 
reducing by a predetermined factor the matrix size of the first and second images prior 

to performing global matching; 

scaling fitted shift values by the predetermined factor for application of the scaled fitted 
shift values to pixels of said one image; 

shifting pixels of said one image using the scaled fitted shift values; 

displaying the subtraction image. 

8. The method of Claim 6, wherein the step of performing image warping comprises: 
reducing by a predetermined factor the matrix size of the first and second images prior 

to performing global matching; 
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scaling fitted shift values by the predetermined factor for application of the scaled fitted 
shift values to pixels of said one image; 

shifting pixels of said one image using the scaled fitted shift values; 
displaying the subtraction image. 

9. The method of Claim 1, ftirther comprising: 

locating locations of candidate abnormalities in the subtraction image; 
extracting at least one predetermined feature fi-om each candidate abnormality; 
comparing each extracted feature with a predetermined threshold; and 
removing firom consideration false positive locations having an extracted feature which 
bears a predetermined relationship with respect to said predetermined threshold. 

10. The method of Claim 9, wherein said extracting step comprises: 

extracting at least one of contrast, standard deviation, and correlation between contrast 
and standard deviation at the locations of the candidate abnormalities in the subtraction image. 

1 1 . The method of Claim 1 0, wherein contrast and standard deviation are extracted. 

12. The method of Claim 10, wherein contrast and correlation are extracted. 

13. The method of Claim 10, wherein standard deviation and correlation are extracted. 

14. The method of Claim II, wherein correlation is extracted. 

15. The method of Claim 14, comprising: 

displaying at least one of said first, second and subtraction images with locations of 
candidate abnormalities under consideration identified in the displayed image. 

16. The method of Claim 3, ftirther comprising: 

locating locations of candidate abnormalities in the subtraction image; 
extracting at least one predetermined feature from each candidate abnormality; 
comparing each extracted feature with a predetermined threshold; and 
removing fi-om consideration false positive locations having an extracted feature which 
bears a predetermined relationship with respect to said predetermined threshold. 

17. The method of Claim 16, wherein said extracting step comprises: 

extracting at least one of contrast, standard deviation, and correlation between contrast 
and standard deviation at the locations of the candidate abnormalities in the subtraction image. 

18. The method of Claim 1 7, wherein contrast and standard deviation are extracted. 

19. The method of Claim 17, wherein contrast and correlation are extracted. 
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20. The method of Claim 1 7, wherein standard deviation and correlation are extracted. 

21. The method of Claim 18, wherein correlation is extracted. 

22. The method of Claim 2 1 , comprising: 

displaying at least one of said original, first, second and subtraction images with locations 
of candidate abnormalities under consideration identified in the displayed image. 

23. The method of Claim 6, further comprising: 

locating locations of candidate abnormalities in the subtraction image; 
extracting at least one predetermined feature from each candidate abnormality; 
comparing each extracted feature with a predetermined threshold; and 
removing from consideration false positive locations having an extracted feature which 
bears a predetermined relationship with respect to said predetermined threshold. 

24. The method of Claim 23, wherein said extracting step comprises: 

extracting at least one of contrast, standard deviation, and correlation between contrast 
and standard deviation at the locations of the candidate abnormalities in the subtraction image. 

25. The method of Claim 24, wherein contrast and standard deviation are extracted. 

26. The method of Claim 24, wherein contrast and correlation are extracted. 

27. The method of Claim 24, wherein standard deviation and correlation are extracted. 

28. The method of Claim 25, wherein correlation is extracted. 

29. The method of Claim 28, comprising: 

displaying at least one of said original, first, second and subtraction images with locations 
of candidate abnormalities under consideration identified in the displayed image. 

30. In a method for computerized processing of at least one chest image, the 
improvement comprising: 

obtaining a digital first image of a chest of a subject; 

detecting ribcage edges on both sides of the lungs in said first chest image; 
determining average horizontal locations of the left and right ribcage edges at plural 
vertical locations; 

fitting the determined average horizontal locations to a straight line to derive a midline; 
rotating the chest image so that the midline is vertical; and 

shifting the rotated image to produce a lateral inclination corrected second image witJi the 
midline centered in said lateral inclination corrected image. 
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3 1 . The method of Claim 30, further comprising: 

obtaining a temporally separated digital third image of the chest of the same subject; 

detecting ribcage edges on both sides of the lungs in said third image; 

determining average horizontal locations of the left and right ribcage edges at plural 
vertical locations in said third image; 

fitting the determined average horizontal locations to a straight line to derive a midline 
in said third image; 

rotating the third image so that the midline detected in said third image is vertical; 

shifting the rotated image to produce a lateral inclination corrected fourth image with the 
midline centered in said fourth image; and 

performing image warping on one of the second and fourth images to produce a warped 
image which is registered to the other of said second and fourth images; and 

subtracting the warped image from the other image to generate a subtraction image. 

32. The method of Claim 3 1 , comprising: 

displaying at least one of the first through fourth and subtraction images. 

33. The method of Claim 30, wherein said step of performing image warping comprises: 
performing global matching to align the lung areas of the one image to the other image; 
performing local matching of selected regions of interest in the right and left lungs of said 

one image to the right and left lungs of the other image to generate shift values for pixels of said 
one image; 

performing surface fitting on the shift values of the right lung of the one image and 
independently performing surface fitting of shift values of the left lung of the one image to 
smooth the shift values and generate fitted shift values for the left and right lungs of the one 
image; 

shifting pixels of the one image using the fitted shift values. 

34. The method of Claim 31, fiirther comprising: 
producing an enhanced subtraction image, including: 
enhancing contrast of the lung regions of the subtraction image; 

adding a predetermined pixel value to the pixel values of the lung regions of the 
subtraction image; 

maintaining pixels of regions other than the lung regions of the subtraction image at one 
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of, 



. (1) a constant pixel value to generate a uniform background, and 
(2) at the values of corresponding pixels in one of the first, second, third and fourth 
images; and 

displaying the enhanced subtraction image. 
35. The method of Claim 33, further comprising: 
producing an enhanced subtraction image, including: 
enhancing contrast of the lung regions of the subtraction image; 

adding a predetermined pixel value to the pixel values of the lung regions of the 
subtraction image; 

maintaining pixels of regions other than the lung regions of the subtraction image at one 



(1) a constant pixel value to generate a uniform background, and 

(2) at the values of corresponding pixels in one of the first, second, third and fourth 
images; and 

displaying the enhanced subtraction image. 

36. The method of Claim 33, wherein the step of performing image warping comprises: 
reducing by a predetermined factor the matrix size of the first and second images prior 

to performing global matching; 

scaling fitted shift values by the predetermined factor for application of the scaled fitted 
shift values to pixels of said one image; 

shifting pixels of said one image using the scaled fitted shift values; 

displaying the subtraction image. 

37. The method of Claim 35, wherein the step of performing image warping comprises: 
reducing by a predetermined factor the matrix size of the first and second images prior 

to performing global matching; 

scaling fitted shift values by the predetemiined factor for application of the scaled fitted 
shift values to pixels of said one image; 

shifting pixels of said one image using the scaled fitted shift values; 

displaying the subtraction image. 

38. The method of Claim 31, fiirther comprising: 



of. 
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locating locations of candidate abnormalities in the subtraction image; 
extracting at least one predetermined feature from each candidate abnormality; 
comparing each extracted feature with a predetermined threshold; and 
removing from consideration false positive locations having an extracted feature which 
bears a predetermined relationship with respect to said predetermined threshold. 

39. The method of Claim 38, wherein said extracting step comprises: 

extracting at least one of contrast, standard deviation, and correlation between contrast 
and standard deviation at the locations of the candidate abnormalities in the subtraction image. 

40. The method of Claim 39, wherein contrast and standard deviation are extracted. 

41 . The method of Claim 39, wherein contrast and correlation are extracted. 

42. The method of Claim 39, wherein standard deviation and correlation are extracted. 

43. The method of Claim 40, wherein correlation is extracted. 

44. The method of Claim 43, comprising: 

displaying at least one of said first, second, third, fourth and subtraction images with 
locations of candidate abnormalities under consideration identified in the displayed image. 

45. The method of Claim 33, further comprising: 

locating locations of candidate abnormalities in the subtraction image; 
extracting at least one predetermined feature from each candidate abnormality; 
comparing each extracted feature with a predetermined threshold; and 
removing from consideration false positive locations having an extracted feature which 
bears a predetermined relationship with respect to said predetermined threshold. 

46. The method of Claim 45, wherein said extracting step comprises: 

extracting at least one of contrast, standard deviation, and correlation between contrast 
and standard deviation at the locations of the candidate abnormalities in the subtraction image. 

47. The method of Claim 46, wherein contrast and standard deviation are extracted. 

48. The method of Claim 46, wherein contrast and correlation are extracted. 

49. The method of Claim 46, wherein standard deviation and correlation are extracted. 

50. The method of Claim 47, wherein correlation is extracted. 

51. The method of Claim 50, comprising: 

displaying at least one of said first, second and subtraction images with locations of 
candidate abnormalities under consideration identified in the displayed image. 
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52. The method of Claim 38, further comprising: 

locating locations of candidate abnormalities in the subtraction image; 
extracting at least one predetermined feature from each candidate abnormality; 
comparing each extracted feature with a predetermined threshold; and 
removing from consideration false positive locations having an extracted feature wliich 
bears a predetermined relationship with respect to said predetermined threshold. 

53. The method of Claim 52, wherein said extracting step comprises: 

extracting at least one of contrast, standard deviation, and correlation between contrast 
and standard deviation at the locations of the candidate abnormalities in the subtraction image. 

54. The method of Claim 53, wherein contrast and standard deviation are extracted. 

55. The method of Claim 53, wherein contrast and correlation are extracted. 

56. The method of Claim 53, wherein standard deviation and correlation are extracted. 

57. The method of Claim 54, wherein correlation is extracted. 

58. The method of Claim 58, comprising: 

displaying at least one of said first, second, third, fourth and subtraction images with 
locations of candidate abnormalities under consideration identified in the displayed image. 

59. An image processing system configured to perform the steps recited in one of 
claims 1-58. 

60. A storage medium storing a program for performing the steps recited in one of 
claims 1-58. 
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